Search engines typically respond to searches by providing search results, which refer to collections of documents, listings of documents, or links to documents, that satisfy a user's specified search criteria. Similarly, the search results page refers to the web page including at least some search results. Typically, the documents that are identified in the listings are ranked according to their relevance to the search query. In addition to the conventional listing of ranked documents, some search engine result pages include a number of sponsored results that are related to the topic of the search query. The sponsored results are paid for by advertisers using a number of methods, and may include fees based on the number of clicks on and position of the sponsored result, as well as monthly or annual subscription fees. The sponsored results are often located in an area of the search results page that is distinct from the unsponsored search results and are not ranked using the same method or algorithm as the unsponsored results. For example, the sponsored result may be located at the very top of the search results page.
The listing of search results and the related sponsored results may come from different sources, such as different databases and/or different search engines. The number of sponsored results that are included in the search results page may vary according to how “commercial” the search query appears to be. For example, if a search query appears to be very commercial in nature (e.g., a search for “flowers and chocolates”), then the number of related sponsored results that are displayed may be six, while if the search query does not appear to be commercial in nature (e.g., a search for “Russia and revolution”) the number of sponsored results that are displayed may be zero.
Commercial Metric
The measurement of how “commercial” search queries are is referred to herein as the “commercial metric” or “C-metric”. The value of the C-metric for a given search query is referred to as the C-level of the search query. The C-level of a query reflects the degree to which the search query is deemed to be “commercial”. Thus, the C-metric corresponds to a graph, while a C-level is a specific value, on the C-metric graph, that corresponds to a given set of inputs.
Various techniques may be used to establish the C-levels for search queries. For example, C-levels may be established by collecting a list of words that are likely to indicate that the user's search is commercial in nature. For example, a person entering a search query including the word “flowers” may very likely be looking to buy flowers. In contrast, a search query consisting of the word “elephant” is not likely to be commercial, because most people do not buy elephants. Once the set of “commercial-indicating” words has been established, the C-value for a given query may be established based on the presence or absence of commercial-indicating words in the query.
Using the presence of commercial-indicating words as the basis for establishing the C-level of queries is only one example of how C-levels may be established. The techniques described herein are not limited to any particular method of measuring the C-metric.
The Quality Metric
In some prior art systems, a quality metric (to be referred to as a Q-metric) is used as a measure of expected customer satisfaction (which may be referred to as an estimated customer satisfaction) with a set of search results. Although not used in the prior art, the Quality level (to be referred to as a Q-level) is the level of satisfaction that a given set of search results provides to the user. The Q-level is measured based on the Q-metric, and the Q-level is a quality or user satisfaction rating. In other words, the distinction between the Q-level and the Q-metric is that the Q-metric is a graph or a table of a mathematical function, while the Q-level is the value of the same function or a point on the graph or table represented by the Q-metric for a given set of inputs to the Q-metric. The Q-metric may be based on the Click-Through Ratio (CTR), which is the ratio of the number of people that click on a link (e.g., an advertising link) in the search results over the number of impressions of the page that contains that link. However, in the prior art, the Q-metric is based on the CTR of the unsponsored search results from a single, purely noncommercial search engine. CTRs may be measured against the CTR of the search results from a noncommercial search engine, using the noncommercial search engine as a standard to which the CTR of the commercial search engine is compared. For example, the CTR of a commercial search engine may be expressed as a percentage of a CTR of a reputable noncommercial search engine, such as perhaps Google, for example.
If too many sponsored results are provided, the user may become frustrated with the experience because it may obscure the search results that the user is interested in. Ultimately, the provider of the search engine would like to maximize its revenues, which may imply maximizing the number of sponsored results on the search result page, the total number of clicks on all sponsored results on all search result pages, the price per click of all or several of the sponsored results on a search result page, or any combination of the above. Consideration must be taken to ensure that the user experience of the search result pages is not adversely affected by the placement, the number, or the quality of the search results on the search results page, whether or not the search results are sponsored or unsponsored.